This application relates to the field of optical communication networks, and particularly to large-scale routers for optical communication networks.
Internet Protocol (IP) packets are unique in the sense that they each have an IP header at the beginning, and they are variable in length. The shortest are on the order of 40 to 64 bytes and the longest can be up to 9600 bytes (known as a xe2x80x9cjumbo framexe2x80x9d length) or potentially longer. In the communications industry today, a standard IP Packet is 1500 bytes maximum length, and a jumbo packet is 9600 bytes. Current non-IP packets include frame relay and ATM packets used with ATM routers. They have different sizes of packets. Some are fixed size but small, they have different characteristics, and their headers have different information in them.
A router is a broad term for a network component that takes information from multiple sources and it routes it to multiple destinations within the network. An IP router takes IP packets specifically and routes them through to various network destinations.
A fabric is a collection of devices which cooperatively provide a general routing capability. One example of a fabric with these properties is a centralized crossbar. Electrical crossbars typically have scalability limitations, limiting their size to tens of ports. In order to scale to larger systems, prior routers have used distributed switching fabrics. These fabrics typically have multiple arbitration points to traverse the fabric, resulting in increased fabric latency and congestion hot spots within the fabric.
An IP router using an optical switch fabric takes in Internet Protocol packets and routes them through an optical switch to various destination ports of the switch. This routing relies on an address location in each packet which is decoded from the packet, compared with a database of address locations, and then used to send the packet on its way to a next router.
Typically, after a packet enters a router, the data and the address of the packet are separated from each other. The address is compared with a look-up table to identify the preferred next router. If that router is available and the links to that router are available and not congested, the router then operates to send the packet to that preferred router. On the other hand, if the preferred next router is clogged, broken, or otherwise unavailable, then the router operates to send the packet through an alternate router over a different path, such that the packet arrives at its original ultimate end location. That path from beginning to the end can be selected from among numerous alternatives. All packets in any particular message need not follow the same paths from beginning to end. The time ordering of packets from a particular source in a network to a selected destination of that network is generally sufficient, such that the efficiency of the end point receiving the packet is maintained.
A problem with present IP routers is that they lack built-in protection mechanisms.
Typically networks contain many routers that have single points of failure. The routers also presently do not scale to large number of ports. Today the largest router is in the order of 8 to 16 ports, which results in very limited configurations that can use these routers. Multiple small routers are built together to make larger configurations. This results in an exponential increase in the number of actual routers compared to the number of ports in the network. It also implies that the latency of data through the multiple stages of routers goes up.
The issue of unreliability or single points of failure in a single router implies that the network overall is less reliable. Any particular router that fails cannot itself contain the error, which must consequently ripple through the wider network. Protocols then must be used to find ultimate routes through the network. Service providers prefer that errors be contained within a small portion of a network, whether it is a link from a source to a destination or a router itself. In this way, a single individual error is contained in a very small portion of an overall network, thereby making the network manageable.
Synchronization of a large distributed system must be accomplished such that the various modules of the system can communicate, without creating any single points of failure. Telecom systems with SONET interfaces typically require clock synchronization such that all SONET interfaces within the entire network are traceable from highly accurate sources.
The present invention is directed to a system and method in which information and control are synchronized as they flow through a large distributed IP router system with independent clocks. The IP router includes multiple equipment racks and shelves, each containing multiple modules. The IP router is based on a passive switching device, which in some embodiments is an optical switching device. However, it is a passive switching device in which the control and the data that go to the switching device come from different sources, which have different clocks. What is described is a timing and synchronization mechanism, such that data and control both arrive at the switching device at the proper time.
The system sends fixed sized chunks of information through the optical switch. A chunk period (also known as a chunk frame period) is on the order of 330 ns in duration and consists of a xe2x80x9cdark periodxe2x80x9d when the optical switches configuration is changed, and a xe2x80x9clight periodxe2x80x9d when a chunk of data passes through the optical switch. The chunk data consists of 400 bytes of payload and roughly 50 bytes of overhead.
The mechanism described accomplishes two types of synchronization. First, the individual clocks for each module in the system are synchronized. Each module in the system is provided a synchronization signal from a centralized clock synchronization shelf. Each module also contains its own crystal clock source that is used if for any reason the clock synchronization distribution fails. The clock crystals on each module are specified to have +/xe2x88x925 ppm (parts per million of frequency variation). In the event of a clock synchronization distribution failure, the individual crystals will have slight frequency differences.
The second type of synchronization required by the system is the propagation of chunk frame period from a centralized location through two different paths to the optical switches. One path is the configuration control for the optical switch and the other path is the chunk data being sent through the optical switch. The lengths of these interconnect paths can be variable over a range from roughly 5 meters to roughly 150 meters.
To accomplish these objectives, a single point in the system originates timing, which is then distributed through various ASICs of the system to deliver configuration control to the switch at the appropriate time. The launch of information to the switch is also controlled with a dynamic feedback loop from an optical switch controller. Control aspects of the optical switch are aligned by this same mechanism to deliver control and data to the optical switch simultaneously.
Various aspects of the invention are described in co-pending, and commonly assigned U.S. application Ser. No. 09/703,057, filed Oct. 31, 2000, entitled xe2x80x9cSystem And Method For IP Router With an Optical Core,xe2x80x9d co-pending, and commonly assigned U.S. application Ser. No. 09/703,056, filed Oct. 31, 2000, entitled xe2x80x9cSystem and Method for Router Central Arbitration,xe2x80x9d co-pending, and commonly assigned U.S. application Ser. No. 09/703,038, filed Oct. 31, 2000, entitled xe2x80x9cSystem and Method for Router Data Aggregation and Delivery,xe2x80x9d co-pending, and commonly assigned U.S. application Ser. No. 09/703,027, filed Oct. 31, 2000, entitled xe2x80x9cRouter Network Protection Using Multiple Facility Interfaces,xe2x80x9d concurrently filed, co-pending, and commonly assigned U.S. application Ser. No. 09/703,043, filed Oct. 31, 2000, entitled xe2x80x9cRouter Line Card Protection Using One-for-N Redundancyxe2x80x9d and co-pending, and commonly assigned U.S. application Ser. No. 09/703,064, filed Oct. 31, 2000, entitled xe2x80x9cRouter Switch Fabric Protection Using Forward Error Correction,xe2x80x9d the disclosures of which are incorporated herein by reference.
The foregoing has outlined rather broadly the features and technical advantages of the present invention in order that the detailed description of the invention that follows may be better understood. Additional features and advantages of the invention will be described hereinafter which form the subject of the claims of the invention. It should be appreciated by those skilled in the art that the conception and specific embodiment disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present invention. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the invention as set forth in the appended claims. The novel features which are believed to be characteristic of the invention, both as to its organization and method of operation, together with further objects and advantages will be better understood from the following description when considered in connection with the accompanying figures. It is to be expressly understood, however, that each of the figures is provided for the purpose of illustration and description only and is not intended as a definition of the limits of the present invention.